Virtual planogram management systems and methods

ABSTRACT

Systems and methods of constructing and managing virtual planograms are presented. Contemplated systems allow for construction of a virtual planogram, which can be used to present consumers virtual inventory items as being available for purchase via a display device. The display device can include an electronic billboard within an establishment, a mall for example, or include a user&#39;s smart device, a cell phone or tablet for example.

This application is a divisional of U.S. application Ser. No.13/667,988, filed on Nov. 2, 2012, which is a divisional application ofSer. No. 13/667,965 filed on Nov. 2, 2012. These and all other extrinsicmaterials discussed herein are incorporated by reference in theirentirety.

FIELD OF THE INVENTION

The field of the invention is virtual planogram management technologies.

BACKGROUND

The number of retailers (e.g., providers of information, goods,services, etc.) has increased over time, leading to the decrease inphysical shelf and display space available per retailer. Thus, retailershave struggled on how to maximize the value of physical retail space.Simply put, retail space is very limited, which restricts the number ofproducts a retailer can place in the space.

With the advent of the Internet, physical space is less of a constraintfor many retailers because they can present their product via a webinterface. For example, retailers can display many products via one ormore images on their web site and allow consumers to purchase thecorresponding product. Moreover, consumers are able to obtaininformation, browse or shop for a physical item, with increasedconvenience and efficiency. Unfortunately, busy consumers require accessto such web sites to enjoy the shopping experience. Busy consumers,especially commuters, don't have time to engage such web sites or don'thave devices that are capable of securely interacting with the web sets.

Tesco™ is one company that has been quite successful in capitalizing onthe idea of selling multiple items to busy consumers through a singlephotograph. Tesco posts a picture of purchasable groceries in subwaystations (e.g., a picture of a Chiquita™ banana that represents allChiquita™ bananas available through Tesco), along with a QR codeassociated with the picture, and allows users to purchase the bananas byscanning the QR code via a smart phone. Unfortunately, Tesco onlypresents static images, a photograph for example, and does not providefor a dynamic representation of available products or product ininventory.

Other companies, such as Poshmark, Inc. and Macy's, have been able tocapitalize on the rise of digital or virtual shopping platforms.Poshmark, Inc., for example, developed an iPhone™ application thatallows users to buy and sell fashion items from their iPhone™ via “realtime shopping events”. Macy's and other retailers have attempted toenhance in-store shopping experiences by providing kiosks that allowcustomers to browse the items of the store they are in via aninteractive screen (See “Touch-screens create online shoppingexperiences at stores,” USA Today, by Hadley Malcolm (Feb. 8, 2012)).

U.S. Pat. No. 8,229,800 to Trotman et al. discloses methods for creatinga virtual shopping area based on one or more constraints. U.S. Pat. No.7,657,457 to Razumov. US Patent Publication No. 2008/0208715 to HOD etal. International Publication No. WO 01/37540 to Perkowski,International Publication No. WO 2012/075589 to Abza, and US PatentPublication No. 2009/0149199 to Maghoul, each attempt to utilize digitalor virtual space for commercial activity. Unfortunately, these also failto provide insight into presenting a dynamic, real-time shoppingexperience for busy consumers.

These and all other extrinsic materials discussed herein areincorporated by reference in their entirety. Where a definition or useof a term in an incorporated reference is inconsistent or contrary tothe definition of that term provided herein, the definition of that termprovided herein applies and the definition of that term in the referencedoes not apply.

However, each of the available platforms, systems and methods fail tomaximize the value of virtual marketplaces for retailers and users.Thus, there is still a need for improved apparatuses, systems, andmethods for providing interactive virtual marketplaces.

SUMMARY OF THE INVENTION

The inventive subject matter provides apparatus, systems and methods inwhich a virtual marketplace (i.e., virtual store space) can be presentedto users based at least in part on virtual planogram(s) that determinehow representations of items should be placed within the marketplace.One aspect of the inventive subject matter comprises an interactivetransaction system, such as an Automated Sales Machine (ASM), having adisplay that presents a virtual item to a user in accordance with avirtual planogram generated by at least one of a planogram manager,planogram server, participating retailer, or management server.

A representation or representations of items (e.g., a virtual inventoryitem, a real inventory item, a virtual service item, etc.) can bepresented in a virtual marketplace on any suitable display. It iscontemplated that the display can be configured to render one or morevirtual marketplaces, including for example, virtual representations ofshelves, malls, stores, windows, racks, revolving racks, hangers, cases,freezers or mannequins, or combinations thereof.

As used herein, the terms “virtual” is used to refer to anything havingcomputer generated imagery (CGI), digital imagery of a real item, videoof a real item, augmented reality, virtual reality, mediated reality, orany other electronic representation of objects that are not physicallypresent. A virtual item would include a superimposed image of areal-world product on an image of a real-world scene. For example, aconsumer could capture an image of a real-world store shelf havingphysically present product. The device can superimpose additionalproducts on the real-world store shelf as a virtual item even thoughtthe virtual item represents a purchasable product that is not physicallypresent.

The items themselves can be virtual or real. Thus, a representation ofan item can be a virtual representation of a virtual service (e.g.,virtual service item), a virtual representation of a virtual inventoryitem (e.g., video game weapon, video game tool, digital song, digitalalbum, digital music video, etc.), a virtual representation of a realinventory item (e.g., food item, clothing item, or any other purchasableproduct, etc.), or a virtual representation of a real service item(i.e., a service provided by an actual human). The items can be itemsoffered by any suitable retailer or retailers, including for example, agrocery store, a jewelry store, a clothing store, an informationalkiosk, a sales kiosk, a small business run from a home, an individualvendor, or combination thereof.

The planogram used in generating a virtual marketplace can be a staticor dynamic in nature. When a display is configured to present a virtualmarketplace according to a static planogram, a representation of an itempresented by the display will not be deleted, moved, replaced, orotherwise modified by a different representation unless the planogram ismodified or replaced. When a display is configured to present a virtualmarketplace according to a planogram that is dynamic, at least onerepresentation of an item presented by the display can be deleted,moved, replaced, or otherwise modified by a different representation,even absent a modification or replacement to the planogram.

It is contemplated that a dynamic planogram can be configured to move,delete, replace, or otherwise modify 1, 5, 10, 100, or even 1000 or morerepresentations upon triggering of an event or events. Contemplatedevents include for example, an item being sold out, a differentrepresentation being moved, a vendor's lease of a space expiring, a timechange, a temperature change, a seasonal change, or any other event thatwould cause a retailer to want a deletion, replacement or othermodification of a representation. Moreover, a planogram can besubstantially or entirely endless in nature, such that a virtual spaceis configured to present some representation at all times, regardless ofthe number of events that take place.

From a methods perspective, a display can be configured to present avirtual marketplace having representations of items according to aplanogram or planograms, simultaneously or sequentially. A planogramcould be generated (e.g., by a planogram manager, display provider,planogram server, and/or a management server, etc.) in a way thatmaximizes profits for him/her/itself. For example, the representationscan be given a placement (i.e., a space) within a planogram based on afee schedule. The fee schedule could be based upon any suitable logic,including for example, auctions, prices that depends on the placementdesired for a representation, prices that depends on a time, weather, orevent, prices that depends on the exclusion of a competitor from avirtual marketplace presented by a display, or prices that depends onthe ability to choose replacement representations (e.g., where thepurchaser can purchase a display or portion thereof, and determine whatreplacements should be made upon an event).

Various objects, features, aspects and advantages of the inventivesubject matter will become more apparent from the following detaileddescription of preferred embodiments, along with the accompanyingdrawing figures in which like numerals represent like components.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is a general overview of an interactive transaction system havinga planogram.

FIG. 2 is a schematic of an interactive transaction system of anestablishment.

FIG. 3 is a schematic of another interactive transaction system.

FIG. 4 is a general overview of a virtual display having a shelf havingrepresentations of items and corresponding symbols.

FIG. 5 is a schematic of a virtual display having a virtual shelf.

FIG. 6 is a schematic of another virtual display having a virtual shelf.

FIG. 7 is a schematic of a virtual planogram.

FIG. 8 is a schematic of a method of initiating a transaction via avirtual shopping experience.

FIG. 9A illustrates a real-world application of a 3D virtual planogramin a physical location.

FIG. 9B illustrates presentation of virtual inventory items within thephysical location of FIG. 9A as dictated by the 3D virtual planogram.

DETAILED DESCRIPTION

It should be noted that while the following description is drawn to acomputer/server based planogram management systems, various alternativeconfigurations are also deemed suitable and may employ varioushardware-based computing devices including servers, interfaces, systems,databases, agents, peers, engines, controllers, or other types ofcomputing devices operating individually or collectively. One shouldappreciate the computing devices comprise a hardware processorconfigured to execute software instructions stored on a tangible,non-transitory computer readable storage medium (e.g., hard drive, solidstate drive. RAM, flash, ROM, etc.). The software instructionspreferably configure the computing device to provide the roles,responsibilities, or other functionality as discussed below with respectto the disclosed apparatus. In especially preferred embodiments, thevarious servers, systems, databases, or interfaces exchange data usingstandardized protocols or algorithms, possibly based on HTTP, HTTPS,AES, public-private key exchanges, web service APIs, known financialtransaction protocols, or other electronic information exchangingmethods. Data exchanges preferably are conducted over a packet-switchednetwork, the Internet, LAN, WAN, VPN, or other type of packet switchednetwork.

One should appreciate that the disclosed techniques provide manyadvantageous technical effects including for example, providing aninteractive transaction system that allows vendors to save costsassociated with physical retail spaces and salespersons by utilizing avirtual space efficiently. The contemplated systems generate virtualplanogram signals, which can be sent to electronic devices. The virtualplanogram signals configure the devices to present purchasable productson a display.

The following discussion provides many example embodiments of theinventive subject matter. Although each embodiment represents a singlecombination of inventive elements, the inventive subject matter isconsidered to include all possible combinations of the disclosedelements. Thus if one embodiment comprises elements A, B, and C, and asecond embodiment comprises elements B and D, then the inventive subjectmatter is also considered to include other remaining combinations of A,B, C, or D, even if not explicitly disclosed.

In FIG. 1, ecosystem 100 represents an interactive transaction systemwhere users 110 can interact with a virtual marketplace via with adisplay 120 and via interface 125. In ecosystem 100, users 110 are ableto purchase products via display 120 even when the products are notphysically present at a store or other commercial venue. The productspresented on display 120 can be presented according to virtual planogram142 thus giving rise to a rich shopping experience. One shouldappreciate that user 110 can interact with the contemplated transactionsystem from nearly any location including an actual brick and mortarstore, a bus station, an airport, a subway station, a mall, at home, orother locations.

Display 120 is configured to present a virtual marketplace(s) to users110 under instruction from product management server 140. The term“virtual marketplace” is used euphemistically to represent a venue ondisplay 120 through which user 110 can conduct a transaction withrespect to one or more items. The virtual marketplace can be digitallyrepresented as an individual product, a collection of products, asection of a shelf, a whole shelf, a display case, a rack, a rotatingrack, a freezer, a store, a collection of stores, a mall, a bazaar, orother forms of venues through which items can be purchased.

Display 120 can include a broad spectrum of devices. In someembodiments, display 120 can include wall-mount high density displays(e.g., HD monitors, computer screens, televisions, etc.) that presentthe virtual market place. In other embodiments, display 120 can beportion of a computing device, for example, display 120 can operate asthe display for a smart phone, a tablet computer, a kiosk, a devicecomposing a kiosk, or comprise any other suitable rendering device.

Interface 125 can also take on many different forms. For example,interface 125 can be multimodal touch sensitive display (e.g., display120, a keyboard, a mouse, or even sensors within the display (e.g.,camera, accelerometer, magnetometer, etc.). User 110 uses interface 120to interact with information presented via display 120.

Display 120 is communicatively coupled to a product management server140 which provides product information or other item information todisplay 120. For example, in a store environment or mall environment,product management server 140 can include inventory information. As user110 interacts with display 120 to purchase products, product managementserver 140 can instruct display 120 to render corresponding productaccording to virtual planogram 142. One should appreciate that productmanagement server 140 can provide product information from one or moreof vendors 145.

Display 120 is also communicatively coupled to transaction engine 130.Transaction engine 130 can be configured to initiate a transaction basedon input received from user 110. Transaction engine 130 can also couplewith product management server 140, through which transaction engine 130can receive salient transaction parameters (e.g., price, quantity. SKU,vendor ID, promotions, account information, security keys, certificates,etc.). When instructed by user 110, transaction engine 130 can initiatea transaction. Example transaction engines can include a point of salesterminal, a local or remote web server, a banking system, or otherdevice configured to interact with display 120. For example, a consumercould interact with display 120 on a tablet computer in a store toidentify a purchasable item that might not be physical present. Theconsumer can then pay for the item at a point of sales device via NearField Communication (NFC).

In the example show, transaction engine 130 and other components ofecosystem communicate among each other via network 135. Example networkscan include the Internet. WAN, LAN, wireless networks, mesh networks,cellular networks, or other forms of communication infrastructure.

Of particular note, display and presentation of purchasable items withindisplay 120 are governed by planogram 142. Virtual planogram 142 is nota traditional planogram representing a physical arrangement of physicalproducts on a shelf. Rather, planogram 142 represents policies or rulesthat govern placement of virtual items within display 120. Thus, theplanogram 142 can be considered virtual in the sense that it defines howpurchasable items are to be displayed in a virtual environment asdepicted within display 120.

Planogram 142 represents a distinct manageable object that is considerseparately manageable from products, goods, services, or items managedby product management server 140. Such an approach is consideredadvantageous as it allows for planogram 142 to be considered a valuablecommodity with respect to product placement as discussed further below.

One or more of planogram 142 can be managed via planogram sever 141. Inthe example shown, planogram sever 141 illustrated as a separate serverthat offers its planogram management services as a for fee service.Entities, possibly one or more vendors 145, can pay a fee (e.g., a bid,a subscription, a flat fee, a per use charge, etc.) in exchange forhaving their products incorporated into planogram 142. Typically,planogram 142 is managed by planogram manager 150. Planogram manager 150can be an owner, vendor, licensee of the planogram, or other entity thatis authorized access to planogram 142.

It is contemplated that a vendor 145 can obtain inclusion in a planogramin various ways. As mentioned, vendor 145 can pay a fee, or vendor 145can have their wares included in planogram 142 simply by being aretailer in a specific establishment (e.g., a mall). Vendor 145 can alsohave their goods or services included by filling out a request form, bya request of the retailer's agent or affiliate, or even through arequest by a third party.

FIG. 2 illustrated a possible embodiment where an interactivetransaction system is integrated within an establishment 200.Contemplated establishments 200 can include: (a) airports, bus stations,train stations, subway stations, taxi stops, or any other transportationhub or public transportation station; (b) indoor malls, outdoor malls,city centers, outlet centers, or any other shopping complex; (c)libraries, museums, amusement parks, or any other publicly accessibleestablishment; (d) travel agencies, tour guide establishments, or anyother business that provides tourism services; and (e) hotels, motels,bed and breakfasts, lodges, cabins, or any other public or semi-publiclodging establishment. Thus, the inventive subject matter is consideredto include construction or placement of a virtual marketplace within oneor more establishments, where the purchasable items are arrangedaccording to the rules within virtual planogram 210.

Establishment 200 can include a device 215 having a display 220 that isconfigured to present a virtual marketplace 221 to consumers. Allcommercially suitable devices configured to present data (e.g.,electronic data, etc.) to a user are contemplated, including forexample, an electronic billboard, kiosk, television, computer screen,tablet computer, and smart phone. For the sake of discussion,establishment 200 can be considered a mall having a plurality ofcomputer monitors (e.g., HD TVs, etc.) operating as display 220. Themonitors can be placed on a wall of a mall corridor and form a wall ofmonitors that display virtual shelves or even virtual stores. Consumersin the mall can interact with the monitors to navigate the stores oreven purchase products. Thus, the virtual marketplace 221 created by themonitors and associated planogram 210 can be considered an AutomatedSales Machine (ASM).

Display 220 can present virtual marketplace 221 in accordance withplanogram 210, which can be generated by planogram server 205. In themall example, planogram 210 can be a free planogram available toparticipating retailers 240 of establishment 200, the stores of the mallfor example. However, it is also contemplated that planogram 210 cancomprise or a for-fee planogram available to retailers 240, or possiblyto other vendors. For example, in FIG. 2, retailer 240 (i.e. vendor) isa retailer of the establishment 200. Still, a remote retailerestablishment 200 could purchase a slot for product placement withinplanogram 210, and thereby present their products via virtualmarketplace 221. Planograms and management thereof will be discussed infurther detail below.

Device 215 can comprise, or be communicatively coupled to (e.g., vianetwork 235 of establishment 200) at least one of interface 225 andtransaction device 230. Each of interface 225 and transaction device 230can be configured to receive at least one of information related to apayment or a user 226, a selection of an item in virtual marketplace221, or inventory of a retailer. Interface 225 can comprise any suitableinterface, including for example, a touch-screen, keyboard, digitalkeyboard, buttons, digital buttons, speaker, voice receiver, andbiometric sensor. Information related to a payment can include anysuitable data, including for example, credit card number, expirationdate, security code, name, address, login, password, phone number, emailaddress, fingerprint, pin number, or other data that can be used forshopping or conducting a transaction.

Consider a scenario where a virtual marketplace 221 is placed within apublic location. One example could include constructing a possibleinteractive transaction system that can be configured to operate as abranded ASM within an airport terminal. For example, the ASM could bebranded as a Trader Joe's™ ASM and be associated with the airport (e.g.,provided by the airport, in a space leased by the airport, pay acommission to the airport, etc.). The airport could install one or moreof display 220 within a terminal and provide planogram 210 to TraderJoe's. In exchange for a fee, the airport can display available productsvia display 220 according to the branded planogram. As travelers passthrough the terminal, they can interact with display 220 to purchaseitems from Trader Joe's. The purchased items can be shipped to the user226 at their home, can be picked up by the user 226 from a near byTrader Joe's on their way home, or can be delivered by a Trader Joe's toan address chosen by user 226. The transaction of purchasing theselected products can be supported by transaction device 230.

One should appreciate that the ASM is not restricted from presentinggoods or services from a single retailer 240 or establishment 200.Planogram 210 can include rules or criteria that dictate placement ofproducts based on many factors. Consider the Trader Joe's example again.The ASM can present products according the planogram 210 where planogram210 is configured to present actual products available from multipleTrader Joe's™ regardless of their locations. For example, planogram 210can dynamically construct an arrangement of products from all thedifferent Trader Joe's based on user preferences, perhaps based on aprotocol defined by the user. A user protocol can include any preferenceor information related to a user, including for example, a sizepreference, a department preference (e.g., having a bakery, deli, etc.),a restriction (e.g., dietary restrictions, allergy, etc.), a brand, avendor, and a (e.g., a specific address, city, or zip code, a state, amaximum distance from the kiosk or other device, distance from a city,etc.). As examples, when user A enters a first geographic protocol(e.g., the zip code 90028) via an interface, the display can beconfigured to present representations of items available at the TraderJoe's on Vine street in Hollywood, Calif., a Trader Joe's local to zipcode 90028. Renderings of symbols associated with the items (e.g., aspecific QR code for a specific brand of chocolate bars, and a differentspecific QR code for a specific brand of veggie chips, etc.). When userB enters a second different geographic protocol (e.g., the city andstate Albuquerque, N. Mex.)), the display can be configured to presentan arrangement of products according to planogram 210 from Trader Joe'slocations in Albuquerque, N. Mex. Still further, planogram 210 canconfigure display 220 to present a virtual representation of an actualretailer. For example, if a user 226 enters the zip code 02115 (i.e.,Boston, Mass.), the display can be configured to present a virtualreplica of the Trader Joe's on Boylston street in Boston. In acircumstance when a store cannot be found to match a user's protocol,geographic or otherwise, a best match can be found. Even where an exactmatch is available, it is contemplated that second best, third best, oreven twentieth or lower best matches can be provided to a user. It iscontemplated that user protocols can be entered via an interface of thedevice, or an interface coupled to but remote to the device.Contemplated interfaces include consumer devices, including keypads,touch-screens, buttons, or any other suitable consumer device.

Another example of a possible interactive system operating as an ASM isan interactive billboard, possibly operating near a duty free shop. Suchan embodiment can allow users to select and purchase perfume, lotion, orother items that cannot be taken with them on flights due to TSAregulations. Such an approach allows travelers to obtain the benefit ofpurchasing tax free goods without requiring them to carry the items on aplane. The interactive billboard (i.e., display 220 and device 215) canpresent representations (e.g., images, video, drawings, names, digitalmodels, etc.) of duty free items available to shoppers flying to aforeign country. Each representation can be associated with a uniquerendered symbol. A shopper can then select certain items by capturing adigital representation of the rendered symbol via a user device (e.g., acamera enabled cellular phone, tablet computer, etc.). When the userwishes to checkout, the user can present a qualifying boarding pass orother verification document to the interactive billboard via averification engine or transaction device (e.g., a barcode or QR codereader of the billboard or user device, etc.). If the transaction deviceor verification engine determines that the user in fact qualifies as aduty-free shopper (i.e., is a qualified user), the user canautomatically have her items ready for pickup at the user's destinationterminal or other destination address (e.g., 10, 100, 1000, or even10,000 miles or more away from the interactive billboard).

Another example of an interactive system that can be provided inestablishment 200 is a kiosk that can allow users to select aparticipating retailer 240 to shop with, including for example,Tiffany's™, Na Hoku™, Conroys™, and See's™. A traveler flying from SanFrancisco to Los Angeles can purchase items, via the kiosk, at the LosAngeles airport. The passenger can be given the option to either pick upthe item at the retailer's store or have the item delivered to anaddress. The kiosk can be configured to print a 3-D version of the itemselected as discussed with respect to FIG. 3, or the kiosk can storeminiature versions of the items selected (e.g., in a storage compartmentof the kiosk) to be made available to purchasers. Thus, one could ordera money clip from Tiffany's to be picked up at a nearby store or otherlocation (e.g., within 10, 50, 100, or 200 feet of the kiosk) selectedby the consumer, as well as obtain a miniature version made of the sameof different material (e.g., paper, plastic, etc.), via the kiosk's 3Dprinter or storage compartment. One can then give the dispensed versionas a proxy to show the recipient what was purchased.

Virtual marketplace 221 presents one or more representations of itemsaccording to planogram 210. The type or format of each representationcan be determined by retailer 240, planogram server 205, planogrammanager, a third party, or other entity assuming proper authorization.The representation can be or comprise an image, a digital model, aseries of non-moving images, motion imagery, a video stream, text, acombination thereof, or any other suitable representation of an item.

Planogram 210 is rendered via planogram sever 205 and can be presentedto the planogram manager according to various formats. For example, theplanogram manager can access planogram sever 205 via a browser interfacein embodiments where planogram sever 205 comprises a web server (i.e.,an HTTP server). Planogram sever 205 can render the planogram 210 forthe planogram manager using different techniques. In some embodiments,planogram 210 can be rendered graphically allowing the manager to dragand drop product representations into available planogram slots. Inother embodiment, planogram 210 can be presented in a tabular form withcorresponding text (e.g., SKUs, GUIDs, retailer information, etc.) orimages. Still, in other embodiments, planogram 210 can render as acomputer model or simulation illustrating a 2D, 3D, 4D (i.e., temporalaspects), or higher dimensional representations of the planogram in theenvironment of establishment 200.

As an example consider a scenario where a planogram manager wishes toinstantiate planogram 210 with respect to toys. The planogram managermight be the toy manufacturer or distributor who wishes to increaseexposure of their toy line when shelf space is limited. The planogrammanager can purchase access to planogram 210. In response, planogramsever 205 provides a planogram template in the form of a 3D modeledshelf as it would appear to a user via display 220. The manager couldplace virtual 3D rendered models of their toys (e.g., dolls, cars,trucks, games, etc.) on to the shelves. Planogram server 205 can, inreal-time, model the physics of the shelf assuming such a feature isdesirable to the manager. Once the manager achieves a desiredconfiguration of toys, the manager can instantiate planogram 210 causingthe planogram sever 205 to make the representations of the toys appearwithin virtual marketplace 221 on display 220. User 226 can thenpurchase or order the toys through interactions with display 220.

User 226 can purchase or otherwise interact with the products managed byplanogram 220 in a number of different ways. In some embodiments,display 220 can include a touch sensitive display. User 226 can initiatea purchase of an item by touching the representation of the item, orsome other portion on the display associated with the item. Additionalinput regarding the purchase can be made by the user through transactiondevice 230 or interface 225. In other embodiments, user 226 can useinterface 225 to indirectly initiate a transaction. For example, user226 can use a camera on their smart phone (i.e., interface 225) tocapture an image of virtual marketplace 221. The cell phone or otherplatform can analyze the image and recognize products on display 220 aspurchasable objects. The camera can then interact with transactiondevice 230 or even device 215 over network 235 to initiate thetransaction. Acceptable techniques for recognizing objects andinitiating transactions are describe in U.S. Pat. Nos. 7,016,532;7,899,243; 7,881,529; and 7,899,252. Once user 226 completes a selectionof items, the user can be presented with advertisements, paymentoptions, and pickup/delivery options.

It is contemplated that the “advertisements” can include suggestions forproducts the user may be interested in, coupons for products, or anyother type of advertisement. The advertisements presented to users canbe based on consumer selections, other information related to the user,random selection, payments made by advertisers, a virtual planogram 210(e.g., space within the planogram is associated with differentadvertisements to be presented to users), or other factors. Paymentoptions can include the option to enter at least one of a credit cardnumber, payment amount, form of payment, authentication or authorizationinformation (e.g., login name, password, security keys or tokens, etc.),address, or other information in support of the transaction.

FIG. 3 is a schematic of an interactive transaction system illustratingan embodiment where the virtual marketplace comprises a visual displayof an interactive virtual shelf. In the example shown, system 300comprises multiple different retailers 301 and 302 who obtain productplacement within planogram 310. For example, a planogram manager 306constructs planogram 310 via a manager interface offered by planogramserver 305. One should appreciate that planogram manager 306 could, infact, be one of retailers 301 or 302. However, it is also possibleplanogram manger 306 could be a third party (e.g., owner of a venue,planogram owner, etc.). Once planogram 310 is instantiated, retailers301 or 302 can engage with planogram sever 305 to ensure their productsare integrated into planogram 301. In the example shown, planogramserver 305 is communicatively coupled with device 315 via network 335.Device 315 comprises a display 320 configured to present virtual shelves350 to user 326. Virtual shelf 350 comprises various representations ofitems 321. User 326 can select a representation, or an item, via displayinterface 325.

The embodiment presented in FIG. 3 allows user 326 to interact with oneor more virtual shelf 350 via device 315 to purchase one or more items.For example, user 326 can utilize a smart phone that constructs virtualshelves 350 according to planogram 310. Virtual shelves 350 comprise avirtual representation or model of a store shelf displaying one or morerepresentations of an item 321 according to planogram 310. One shouldnote that virtual shelf 350 can include products from one or more ofretailers 301, 302, or other retailers. In some embodiment virtual shelf350 can reflect actual inventory of retailers 301 or 302, or even anactual shelf in the establishments of the retailers. Regardless of theform taken by virtual shelf 350, user 326 can interact with the shelvesin a virtual fashion.

User 326 has numerous possible points of interactions with virtual shelf350 via interface 325. For example, user 326 can browse across shelf350, pan device 315 across shelf 350, select or zoom into items 321, orotherwise shop for items 321 as permitted by planogram 310. One shouldappreciate that virtual shelves 350 could be represented as being largerthan the field of view permitted by display 320. In such scenarios,panning device 315 (e.g., moving device left, right, up, down, etc.)allows user 326 to observer other portions of shelves 350. This can beachieved through monitoring sensor data from device 315. In someembodiments, accelerometer or magnetometer data can be used to derive aposition or orientation of device 315 through properly integrating thesensor data. Additionally, device 315 can leverage Visual SimultaneousLocalization and Mapping (vSLAM™; Karsson et al. “The vSLAM Algorithmfor Robust Localization and Mapping”, Proc. of Int. Conf. on Roboticsand Automation (IRCA) 2005) technology to determine position ororientation of device 315. When desire, user 326 can initiate atransaction associated with one or more of items 321 via transactiondevice 330, possibly a financial transaction with a bank as illustrated.

In some embodiments, system 300 can include 3D printer 345. Once asuitable transaction is complete for an item 321, user 326 couldpossible download a file comprising instructions or code for 3D printer345 that defines a rendering of selected item 321. For example, the filecould be in STL, VRML, or WRL file formats describing the 3D nature ofthe selected object. One should appreciate that printed object could bethe item actually purchased, or could be a proxy item made available touser 326 until the actual product arrives. Such an approach isconsidered advantageous in embodiments where user 326 requires immediateaccess to an item or wishes to have instant gratification; perhaps item321 is a toy that requires shipping but the proxy object can be playedwith immediately. All commercially suitable three dimensional printersare contemplated, including for example, printers made by Solidoodle, 3DSystems (e.g., ProJet 1500, Cube 3D, RapMan, etc.), PP3DP Company,LeapFrog (e.g., Creatr, Xeed, etc.), Bits from Bytes, Ultimaking Ltd.,MaukCC, MakerBot Industries, Magicfirm LLC, FormLabs, Asiga (FreeformPico, Pico Plus, etc.), or Stratasys, 3D printer 345 can be configuredto print three dimensional items or samples of an item (e.g. a miniatureversion, less expensive version to try on for size, etc.).

Transaction device 330 allows user 326 to enter information related toat least one of a payment or a user selection. In some embodiments,users can additionally or alternatively enter information related to apayment via consumer interface 325. Transaction device 330 and/orinterface 325 will then communicate with a bank or other financialinstitution via network 335, to initiate a transaction (e.g., an order,purchase, authorization, verification, etc.) associated with a consumerselection through one or more protocols (e.g., FIX protocol, SecureElectronic Transaction, RFC 1898, etc.).

In FIG. 4, ecosystem 400 represents a transaction system providingadditional insight into management of virtual shelves 460 and 461 in avirtual marketplace as rendered on display 420. Further, in theembodiment illustrated, aspects of ecosystem 400 are combined. Forexample, planogram sever 441, transaction engine 460, and database 455are integrated into single product management server 440. Suchapproaches are advantageous in scenarios where an establishment, a mallfor example, wishes to retain control over ecosystem 400. One or more ofvendors 445 can interact with planogram 442 via a vendor interface (notshown). Planogram manager 450 manages planogram 442 via a managerinterface (also not shown).

Database 455 can be configured to store one or more data objects relatedto ecosystem 400. More specifically, database 455 can store planogramobjects, associated product information, account information (e.g.,vendor accounts, consumer accounts, planogram manager accounts, etc.),prices, auction information, or other types of objects.

As previously discussed, planogram 442 determines which representationsof items 463 are presented and according to what arrangement withindisplay 420. In this example, display 420 presents two virtual shelves460 and 461 similar in nature to that described in FIG. 4 with theaddition of symbol 462. Planogram 442 can include instructions thatconfigure display 420 to render one or more symbol 462. Symbol 462 canbe used to aid in determining the identity of items 463. For example,symbol 462 can be used to identify one or more vendors 445, a specificshelf, a product, a product model, an establishment, a location ofinformation, or other indicia. For example, planogram 442 could instructdisplay 420 to present a QR code that specifically includes a SKU foritem 463 and a URL pointed to transaction engine 460. In such cases, auser can leverage a QR code reader on their camera-enabled cell phone toinitiate a transaction with respect to one or more of item 463. Oneshould appreciate that symbol 462 is considered an electronic ordigitally displayed symbol rather than a symbol on printed media. Thus,symbol 462 can change subject to conditions dictated by planogram 442.Example symbols can include bar codes, matrix codes, alphanumeric codes,text, logos, watermarks, or other. Further, symbols 462 can includenon-visual symbols possibly including NFC, dynamically changing RFIDtags, audio signatures (e.g., audible or non-audible, etc.), or otherforms of indicia.

FIG. 5 represents an environment where a virtual marketplace isconstructed according to planogram 510, which is instantiated byplanogram server 505. In this example, representations of items 521include products form multiple retailers 501, 502, or others and arepresented via display 520 per instructions defined within planogram 510.Each item 521 includes a symbol 522 (e.g., bar codes, matrix 2D barcodes, stacked bar codes, high capacity color bar codes, data glyphs, QRcodes, maxicodes, etc.) so that each item 521 can be easily identified.User 526 interacts with display 520 and with items 521 via a sensorenabled device 515 and device interface 525. For example, user 526 canenter at least one of a selection of an item, a selection of arepresentation, a selection of a symbol, a quantity, a size, a price, arange thereof, a delivery address, or other information related to atransaction. Device 515 or display 520 can then present a question,response, confirmation, or any other suitable reply to user 526 toinitiate or complete the transaction.

In the embodiment shown, the virtual marketplace is communicativelycoupled with transaction manager 555 configured to aid in completing atransaction. For example, transaction manager 555 can be configured toverify payment, confirm payment, initiate delivery to a location (e.g.,30, 200, 1000, or even more miles away from the transaction device,etc.), or other actions that aid in completing a transaction. Anacceptable system capable providing such services include Amazon™ orPaypal™.

One example of a transaction system that could leverage use of symbols522 includes an electronic billboard of a train station or otherestablishment. The electronic billboard can be configured to present avideo representation of purchasable products of a grocery store ormultiple grocery stores, along with QR codes or other symbolscorresponding with the purchasable products, on different virtualshelves/marketplaces. In other embodiments, the display can presentstill image representations of items 521, which can be updated on aperiodic basis, for example, possibly every one, five, or ten minutes asgoverned by planogram 510. Each shelf could be presented according tothe same or different planogram 510. For example, each shelf could begoverned by a different instance of the same planogram 510. It iscontemplated that a video representation of items 521 can be a livevideo stream of the products as they are presented in the physicalgrocery store. Alternatively, the video representations can bepre-recorded or computer generated. It is also contemplated that thevirtual marketplace can be navigable such that the user can determinewhat shelf to view, what portion of a marketplace to view, whatrepresentations to view, or section of a shelf to view. A user awaitinga train can thus navigate through a virtual marketplace by interactingdirectly with interactive display 520 of the billboard, and order,reserve, purchase, obtain information related to or even comment on anitem represented. To initiate any of these transactions, the user cansimply capture an image of the QR code or other symbol via their cameraenabled phone. This approach is considered advantageous becauseconsumers can interact directly with the billboard for an immersiveshopping experience while ensuring transactions remain secure throughtheir personal device. Once items have been selected, the user can enterpayment and delivery information via one or more interfaces andtransaction devices, and have the items delivered to their home beforethey get there.

One should appreciate that symbols 522 can represent actual inventoriedproducts located at retailer 501 or 502 physical location. In suchcases, the system can allow retailers to present inventory items toconsumers without having to occupy large amounts of physical space.Rather, the items 521 can be stored in an inventory holding room wherepresentation is not as important, and space is more readily available.It is contemplated that each display can have a fingerprint or otherbiometric sensor that allows a user to be authenticated or identified bypressing their finger against the sensor. Once the user is authenticatedor identified, the user can use a touch-pad or other interface of adevice to select items for purchase. Once the user pays for the items,an employee of the physical marketplace can gather the selected items,package them, and send them with the user before they leave the physicalmarketplace. Again, the user can capture sensor data representing symbol522 to initiate a transaction.

Yet another embodiment includes a kiosk having virtual navigationcapabilities within a mall. For example, the kiosk can be configured todisplay a navigable representation of a shopping complex or portionthereof. Retailers within the mall can pay a fee to reserve a space fortheir products within a virtual marketplace governed by planogram 510.The virtual marketplace can comprise virtual representations of thephysical mall in digital model (e.g., 3-D) format, but be limited torepresentations associated with participating retailers, whilenon-participating retailers are shown as a blank space, 2-D format, orother suitable differentiating representation. With the display beingnavigable, it is contemplated that the user can select a preferred viewhe wishes to have of the mall. For example, the user can select a viewshowing an avatar representing a user walk through different portions ofthe virtual marketplace, entering and exiting various wings of the mall,and various stores. Alternatively or additionally, the user can select aview that allows a user to view portions of the mall as if they wereactually in that space. Thus, the user can navigate the virtualmarketplace and feel as though they are actually walking around thephysical mall.

When the user enters a representation of a store via the virtualmarketplace, the user can browse and select items to purchase as theywould be displayed in display 520 according to planogram 510. Each item521 can represent an actual item in the corresponding store of retailer501 or 502. If the user wants to feel a fabric, see what the item wouldlook like on them, or have any other physical experience with the item,it is contemplated that the kiosk can comprise at least one of a 3-Dprinter that is configured to print versions of the items (e.g., plasticor metal samples), a vending machine configured to dispose samples tousers upon request (e.g., lipstick samples, fabric swatches, perfumesamples, etc.), and a device that scans an outer perimeter of a user,determines what sizes are most appropriate for that user and allows theuser to see what an item would look like on them. Additional informationregarding 3-D printing can be found in “Yes, the future of retail isupon us.” by Jon Swartz, USA Today (Aug. 9, 2012). The user can interactwith a management server and transaction engine to initiate atransaction related to a representation of an item selected. Thisinteraction can occur over a network of the establishment, or any othersuitable network.

Motels, hotels, and other lodging establishments can provide displays toguests that allow them to purchase items from remote retailers sellingvarious classes of goods. For example, a display can be configured topresent representations of items from one retailer dealing with oneclass of goods or services, and representations of items from adifferent retailer dealing with a different class or classes of goods orservices, including for example, grocers, jewelry stores, consumerelectronics retailers, ticket (e.g., movie, concert, play, etc.)vendors, tourist services vendors, massage service providers, and soforth. As used herein, a “class” is a category of items having someattribute in common and differentiated from others by kind, type, orquality. Contemplated classes include generic product classes, includingfor example, aspirin, paper towels, sugar, and so forth. It iscontemplated that any suitable classification scheme can be used todetermine a class, including for example, the InternationalClassification of Goods and Services associated with the United StatesPatent and Trademark Office, the ISIC classification scheme, and schemesbased on brand names, types of products, or names of products. Providingthis type of display to guests can attract them to the lodgingestablishment because of the increase in convenience.

It is contemplated that different retailers of a planogram or virtualmarketplace can deal with the same or different classes of goods. It isalso contemplated that the retailers can deal with the same class ofgoods when one scheme applies (e.g., brand), and different classes ofgoods when another scheme applies (e.g., type of product). Moreover, theretailers can deal with the same class of goods even when two or moreschemes apply (e.g., brand, type of product, etc.).

The display(s) provided by the lodging establishment can be configuredto present virtual representations of various shelves, each onecomprising virtual representations of real inventory items according todifferent planograms. For example, a first shelf can be provided inaccordance with a first planogram and comprise representations of WholeFood's™ real inventory items (e.g., vitamins, etc.), while a secondshelf can be provided in accordance with a second planogram and compriserepresentations of Target's™ real inventory items (e.g., clothing,etc.). Moreover, a first rotating rack can be provided in accordancewith a third planogram and comprise representations of Nordstrom's™ realinventory items, while a third shelf can be provided in accordance witha fourth box and text planogram and comprise representations of servicesmade available by Burke Williams™. The retailers participating in avirtual marketplace can be, but do not need to be, located in closeproximity to one another, or even deal in the same classes or brands ofgoods and services. Thus, the Whole Food's can be located in Brooklyn.N.Y., while the Burke Williams is in Silverlake, Calif.

FIG. 6 illustrates the varied nature of how items can be represented indisplay 600 according to dynamic planograms where a dynamic planogram isconsidered to be a planogram capable of changing with time or changingbased on triggering conditions (e.g., news events, user preferences,retailers, etc.). Display 600 comprises a virtual shelf having threecolumns. The first column presents a representation of a first item 660and a second item 661, having corresponding renderings of first andsecond symbols. 670 and 671, respectively. The first column illustratesthat a planogram can govern which products are placed relative toanother product or products. In this case, the consumer sees item 660placed on the shelf in front of item 661. Similarly symbol 670 appearsin front of symbol 671. One should appreciate that items 661 might notnecessarily be visible to a consumer until all items 660 have beenpurchased. Such an approach is considered useful to create value withinthe planogram or possibly to allow a vendor to virtually rotate stock.For example, item 660 could represent older stock where symbol 670represents the SKU for the other stock item.

The second column comprises a representation of a third item 662 and afourth item 663, along with a rendering of a third symbol 672 that isassociated with both the third and fourth items. The user can bepresented with only a representation of a third item 662 until the itemruns our or is otherwise not available, at which point therepresentation of a fourth item 663 can be presented to a user.Additionally or alternatively, representation of a third item 662 can bepresented to a user wishing to order the item in a size 6. If that sizeis not available, the user can then be presented with a representationof a fourth item 663 different from third item 662, which is availablein a size 6. The third column comprises a representation of a fifth item664, and a corresponding rendering of a fourth symbol 673 associatedwith the fifth item.

A static or dynamic planogram that can be used to generate a virtualmarketplace can be based on a template planogram, possibly where thetemplate corresponds to a real planogram used in an actual store topresent product displays on fixtures (e.g., shelves, malls, stores,windows, racks, revolving racks, hangers, cases, freezers or mannequins,or combinations thereof, etc.). Thus, a virtual marketplace can beconfigured to display the items that are present in a physicalmarketplace, and update according to the changes made in the physicalmarketplace. For example, the planogram can represent a physical freezerin a grocery store that has, from left to right, 5 CPK™ frozen pizzas, 3Hot Pockets™, and 7 Marie Calendars™ chicken pot pies. A dynamicplanogram can be used to generate a corresponding virtual freezer ondisplay 600 available inside a grocery store, or remote to, the grocerystore. This virtual freezer would comprise, from left to right,representations of 5 CPK™ frozen pizzas, 3 Hot Pockets™, and 7 MarieCalendars™ chicken pot pies. If a frozen pizza is removed from thephysical freezer (e.g., for purchase by a shopper, relocation, etc.),the frozen pizza can be removed from the virtual freezer in real-time orsubstantially real-time. If the planogram used to display items in thefreezer is configured to instruct the replacement of the frozen pizzawith a strawberry fruit bar, the dynamic planogram used to generate thevirtual freezer can configure the display to present a representation ofa strawberry fruit bar in place of the frozen pizza, also in real timeor substantially real-time.

FIG. 7 illustrates construction of virtual planogram 770. Planogrammanager 710 interacts with planogram sever 720 to create, update,instantiate, or otherwise manage virtual planogram 770. Planogram server720 generates a planogram 770 in accordance with information provided byplanogram manager 710, possibly based on input provided by vendors 730,740, 750, 760, or others. It is contemplated that planogram manager 710establishes the nature of planogram 770 including its capacity,arrangement, pricing, or other attributes. For example, planogrammanager 710 could include a virtual representation of a mall and couldcreate planogram 770 to function as a part of an ASM for the mall, whichoffers products physically available at the mall's many retailers. Theretailers can then pay for slots within planogram 770. Information usedto create planogram 770 can include rules governing absolute or relativeplacement of products in slots, rules governing planogram behavior basedon triggering criteria (e.g., events, user indent, inventory, time,location, etc.), conditional rules (e.g., if-then-else, logicaloperators, etc.), or other rules.

It is contemplated that planogram 770 can be n-dimensional (for physicaland logical dimensions), and thus representations of items can be placedwithin a position of a virtual marketplace according to one, two, ten,fifty, or even a hundred or more of the following: a time, a location, anews event, a temperature, a fee, a trend, a weather condition, a priorpurchase, a user preference, a category, or any other event orcondition. In the example shown, planogram is euphemistically presentedas a 2D planogram of a shelf divided into horizontal slots and shelfdepths. However, one should appreciate that planogram 770 can have many“physical” dimensions (e.g., shelf position, position within shelf,angle a representation is placed within a shelf, etc.) or many logicaldimension (e.g., time, trends, event, demographics, etc.). Thus slotswithin planogram 770 can be characterized according to some or all ofthe dimensions and priced accordingly, possibly based on differentweights. Vendors 730, 740, 750, and 760 can each purchase slots withinplanogram 770 by paying a fee to planogram manager 710 or the serviceoperating planogram sever 720. The fee required for placement can dependon one or more factors, including for example, a prior purchase ofslots, desirability of a slot, a time of purchase, popularity of a slot,dynamic nature of a slot, physical dimension, logical dimension, size ofa slot, exclusion of others from a slot, slots, or planogram, acontinuous presence of a representation despite a user or other protocoldictating otherwise, or any other suitable factor. For example, afor-fee planogram can comprise a product placement policy based on a feeschedule.

As shown in FIG. 7, dollar symbols represent a cost for a slot, where asingle dollar sign represents a low cost, and more dollar signsrepresent higher and higher costs. Column one rows four and five show avirtual marketplace slot that is twice as large as the standard slot.Thus, Vendor 1 can place double the number of representations or itemsin this slot as Vendor 1 could in an adjacent slot.

First column, second row shows a slot that has been excluded by Vendor5. This means that Vendor 5 paid a fee to have no items presented nextto its product A (Column two, row two). Column five, row two shows aslot that is available for purchase at a low price. This slot is twiceas large as the standard virtual marketplace space but available at alower price than some smaller slots, perhaps due to its location on theright side of the shelf, its proximity to another product or vendor,having been available for a long period of time, or some other factor.

The slot in column three, row three is also available, but at differentcosts depending on what vendor purchases the slot. If vendor 1 purchasesthe slot, it will be cheaper, perhaps because Vendor 1 has alreadypurchased so many of the slots within the virtual marketplace.

Each slot within a planogram 770 can be sold for any suitable period oftime. Thus, a slot can be sold on an hourly or daily basis, or can evenbe sold for months or years at a time. It is also contemplated thatmanagement options for a slot or slots within a virtual marketplace canbe sold. For example, a vendor or group of vendors can purchase columnone, row four of a planogram 770 for a one year period for $10,000 USD.This purchase can come with an option for direct management of the slotfor an additional $1,000 USD. This can allow a vendor to manage the slotwithin the planogram possibly as a proxy for planogram manager 710. Thiswould allow the vendor to perform various actions and customize a slot,including for example, including multiple representations within theslot, changing the representation at their whim, deleting arepresentation (e.g., as inventory decreases, to create a buzz, etc.),placing an advertisement within the slot, selling advertising spacewithin the slot, and choosing the format of the representation (e.g.,still imagery, motion imagery, blinking imagery, sound, a combinationthereof, etc.).

It is contemplated that each “slot” within a virtual marketplace asgoverned by planogram 770 can be configured to present at least one of asingle representation, multiple representations, or even a virtualmarketplace (e.g., rotating rack that is rotatable via touch-screen,button, etc., shelf, etc.) within the virtual marketplace.

In FIG. 8, method 800 illustrates a method of initiating a transactionvia a virtual shopping experience. Method 800 include step 810 ofproviding access to a display (e.g., a computer controllable display,etc.). Providing access to a display can take on many different formsdepending on the nature of the desired shopping experience. In someembodiments, the display can be installed as electronic bulletin boardswithin an establishment, a mall in a pre-operational stage, or existingshopping center as suggested by step 815 for example. In otherembodiments the display could include a hand held device display. Itshould be appreciated that providing access to a display could alsoinclude allowing consumers to install an application on their smartdevices so that the display can be configured via a planogram server asdiscussed above.

Step 820 can include configuring the display, possibly via a planogramserver, to present a first virtual shelf comprising first and secondvirtual inventory items according to a planogram. The virtual inventoryitems are considered to include available items for purchase. Such itemsdo not necessarily have to be physically present with the user or thedisplay. For example, a planogram manager can access the planogram severto construct or otherwise manage the planogram via a planogrammanagement interface (e.g., browser, API, HTTP server, etc.). Asdiscussed above, the planogram represents a virtual data construct orobject that dictates how items are to be displayed according to one ormore rules or factors. Example factors that can be used to govern anarrangement of the items can include relative positions, absolutepositions, location, user preferences, vendor preferences, establishmentattributes, augmented or virtual reality elements, time, events, orother factors. For example, the planogram can include rules that governplacement of virtual items (i.e., images of real-world objects) on ashelf in an augmented reality view on the display relative to itemsactually present on the shelf.

Step 830 includes allowing a user to select at least one of the virtualinventory items represented on the display. One should appreciate thatthe user can also select an item that is physically present via thedisplay as well. As an example, consider a scenario where a usercaptures an image of a store shelf populated with a specific brand ofproducts with their camera-enabled tablet computer (e.g., smart phone,iPhone, Galaxy tablet, etc.). The image could be a still image, videodata, or even a live real-time image. The tablet's display can use theimage data to derive image characteristics, which can then be used torecognize the product brand. In response, the tablet can obtain avirtual planogram from the planogram sever where the planogram isconfigured to allow presentation of the existing physically presentproducts along with additional virtual items superimposed on the imageand neighboring images of the physically present items. Thus, thedisplay presents a virtual or augmented reality shopping experienceaccording to the planogram. One should appreciate that the virtualinventory items, which can be real-world products available forpurchase, could be alternative products from the same brand, be productsfrom alternative brands, include sales items, include promotion items,or include other types of items.

Step 840 includes configuring a transaction device to receive accountdata associated with the user and related to the selected inventoryitem. The account data can include a broad spectrum of information andcan depend on the nature of the transaction. For financial transactions,the account data could include bank account identifier, credit cardnumbers, security tokens, price information, number of units topurchase, SKUs, or other information. For non-financial transactions,the account data could include network addresses, user name, passwords,security keys (e.g., public encryption key, etc.), or other types ofinformation sometimes required for the non-financial transaction.Example non-financial transactions could include making reservations(e.g., hotel, travel, restaurant, etc.), making appointments,interactions among software packages, protocol transactions, a check-in,or other types of interactions.

Step 850 includes initiating a transaction (e.g., over a network), viathe transaction device, using a transaction protocol based on theaccount data. One should appreciate that a transaction can be initiatedby the user's device, through a point of sales terminal (e.g., cashregister, credit card reader. NFC device, bar code scanner, etc.), aback end product management server, the planogram server, or othercomponent within the ecosystem. In some embodiments, the fulltransaction is carried out by the transaction device while in otherembodiments the full transaction can be carried out by one or more otherdevices, or the transaction device and one or more other devices. Forexample, a person can enter their account information into their cellphone, which in turn interactions with a point-of-sales device via NFC.The point-of-sales device communicates with one or more back end serversoperated by a bank or credit card company. Confirmation for thetransaction is then sent back to the point-of-sales device and thenpresented to the person on their cell phone via the NFC communicationslink.

Purchasing a virtual inventory item via the display does not necessarilyresult in the purchaser receiving a physical product at the point ofpurchase. Therefore in some scenarios, as suggested by step 860, themethods can include initiating a delivery of the selected inventory itemto a destination location, possibly selected by the user. For example,the account data or other user data provided to the transaction devicecan include a shipping address. When the transaction is initiated, theshipping address can be forwarded to the fulfillment component of thetransaction system. The fulfillment component could include a warehouse,vendor, retailer, on-line site (e.g., Amazon, Ebay, etc.), or othercomponent capable of shipping the physical products. In someembodiments, the physical item does not yet exist. As discussedpreviously, one aspect of delivery of the inventory item can includeprinting the item or a proxy of the item for the purchaser. The item orproxy could be printed using a 3D printer.

FIGS. 9A and 9B illustrate a real-world application of a virtualapplication in a physical real-world location. The photographs withinFIGS. 9A and 9B are frames from a proprietary video illustrating use ofthe inventive subject matter. In FIG. 9A a person uses camera-enableddevice 915, a tablet computer, to capture an image of a real-world shelf923 having a collection of physically present items, toy dolls in thisexample. There are several points of note. First, physical shelf 923lacks space for additional items. Second, the shelf displays items froma specific brand (e.g., Disney®). Third, although difficult to see, justright of the shelf is another aisle in the physical location. Althoughthe physical location could be an actual retailer establishment, in thisexample the physical location is a private location, that is, anon-public location, constructed for prototyping the inventive subjectmatter.

Device 915 as illustrated is instrumented with multiple aspects of theinventive subject matter. More specifically, device 915 comprises arecognition module configured to covert the image data into a one ormore image features (e.g., SIFT, BRISK, vSLAM, etc.) that can be used torecognize shelf 923 or the physically present items. Device 915 uses theimage features to select one or more virtual planograms over a networkconnection from a planogram server. One should appreciate that theplanogram server functionally could be deployed within device 915 as anapp where the planogram server could pre cache relevant virtualplanograms within device 915 based on device GPS location or otherfactors. Regardless of the source of the virtual planogram, as theperson begins panning to the right, the virtual planogram causes device915 to present products accordingly.

In FIG. 9B, device 915 has been panned to the right. The right edge ofreal-world shelf 923 is visible at the left of the frame. Further,between the right edge of real-world shelf 923 and the left edge ofdevice 915 one can see the vacant aisle within the physical location.However, device 915 does not present an image of the aisle. Rather,device 915 presents an image of a portion of shelf 923 and images ofvirtual inventory items 921. Virtual inventory items 921 are placedrelative to shelf 923 and include additional purchasable items that arenot physically present in the physical location. Placement of virtualinventory items 921 are arranged according to the virtual planogramtriggered by the image features derived from the image of real-worldshelf 923. The person can now interact with virtual inventory items 921by simply tapping on a desired product.

One should appreciate that virtual inventory items 921 are presented ona virtual shelf, which can be considered to virtually expand the shelfspace in the physical location by ten, twenty, fifty, a hundred, or evena thousand or more times. In addition, real-world shelf 923 onlypresents a single set of toys associated with a single franchise (e.g.,Disney baby princesses) while virtual inventory items 921 allows forexpanding on the products available to the person by offering productsfrom another franchise (e.g., Brave® movie products). Still further, theshopping experience of the user is expanded to traditionally unavailablespace, for example, the aisle space in this embodiment.

Although not illustrated, the virtual planogram controls presentation ofitems 921 in a full 360 degree experience. As the person continues topan around beyond the Brave display, additional items become visible.This can be achieved by device 915 registering background features fromthe image data and combining the registered background features withinertial movement of the device from accelerometer data obtained from anaccelerometer. The combined data sets can then be used to generate highprecision position and orientation information of device 915 so thatvirtual inventory items 921 are displayed properly where they aresuperimposed on irrelevant background items (e.g., the aisle), while notoverlapping with relevant items (e.g., physically present items onreal-world shelf 923). Such precision can be obtained throughapplication of the previously mentioned vSLAM algorithms. It should beappreciated that such control of placement is governed by the rules orpolicies of the virtual planogram. Thus, the virtual planogram cancontrol arrangement of visible or non-visible items as registered byphysical features in the physical location.

The approach illustrated in FIGS. 9A and 9B gives rise to numerousadvantageous. One advantage includes allowing virtual planograms tooverlap. For example, the user could image a set of neighboringproducts, which can trigger a new virtual planogram that causespresentation of a new set of virtual inventory items in place of virtualinventory items 921. Thus, the virtual planograms can be considered tobe layered or overlapping where the active planogram is triggered by thecurrent sensor data. Another advantage includes allowing the retailer toparticipate in sales of virtual inventory items 921. If the userpurchases one of items 921, the retailer at the physical location can becredited even though they did not have the item present. Yet anotheradvantage includes allowing the planogram manager to dynamicallyinteract with the user via the virtual planogram in real-time. Forexample, the planogram manager can grant “award” points to the purchaserupon purchase, speak or chat directly with the purchaser, request surveyinformation, or have other types of interaction(s). Still further,virtual items 921 can be changed according to the current circumstancespossibly based on user preferences, proximity of other users,geo-location, time, news events, or other factors.

Another example use case could include construction of a virtual storeshelf based on available products offered via other retailers. Forexample, products offered by Amazon, eBay, Barnes and Noble, or otheron-line retailers can be aggregated together or individually to giverise to a virtual shopping experience. Consider eBay as an example. Aperson could use their device to image a collectable action figure. Inresponse, the device can contact a planogram sever, which in turn usesthe image data to construct, in real-time, a virtual planogram ofsimilar action figures available on eBay, possibly based on a templateplanogram. Such an approach allows a user to comparison shop in realtime. Thus, the inventive subject matter is considered to includeconstructing a virtual planogram in sustainably real-time based onsearch results that are responsive to a query generated on contextualsensor data.

Unless the context dictates the contrary, all ranges set forth hereinshould be interpreted as being inclusive of their endpoints, andopen-ended ranges should be interpreted to include commerciallypractical values. Similarly, all lists of values should be considered asinclusive of intermediate values unless the context indicates thecontrary.

As used herein, and unless the context dictates otherwise, the term“coupled to” is intended to include both direct coupling (in which twoelements that are coupled to each other contact each other) and indirectcoupling (in which at least one additional element is located betweenthe two elements). Therefore, the terms “coupled to” and “coupled with”are used synonymously. The terms “coupled to” and “coupled with” areused euphemistically to mean “communicatively coupled with” where two ormore elements are able to communicate with each other over a networkpossibly via one or more intermediary network devices.

As used in the description herein and throughout the claims that follow,the meaning of “a,” “an,” and “the” includes plural reference unless thecontext clearly dictates otherwise. Also, as used in the descriptionherein, the meaning of “in” includes “in” and “on” unless the contextclearly dictates otherwise.

It should be apparent to those skilled in the art that many moremodifications besides those already described are possible withoutdeparting from the inventive concepts herein. The inventive subjectmatter, therefore, is not to be restricted except in the scope of theappended claims. Moreover, in interpreting both the specification andthe claims, all terms should be interpreted in the broadest possiblemanner consistent with the context. In particular, the terms “comprises”and “comprising” should be interpreted as referring to elements,components, or steps in a non-exclusive manner, indicating that thereferenced elements, components, or steps may be present, or utilized,or combined with other elements, components, or steps that are notexpressly referenced. Where the specification claims refers to at leastone of something selected from the group consisting of A, B, C . . . andN, the text should be interpreted as requiring only one element from thegroup, not A plus N, or B plus N, etc.

1-21. (canceled)
 22. A virtual planogram server comprising: a tangible,non-transitory computer readable memory storing software instructions;at least one processor coupled with the memory and that is, uponexecution of the software instructions, configurable to: receive adigital representation of a physical store fixture having physicallypresent products; convert the digital representation into digitalfeatures associated with the physical store fixture; obtain a virtualplanogram, via a planogram database, selected based on the digitalfeatures, wherein the selected virtual planogram includes a virtualproduct display fixture and comprises an expansion of the physical storefixture to unavailable real-world space relative to the physical storefixture; generate a virtual market place from the virtual planogram andhaving the virtual product display fixture rendered relative to thephysical store fixture, wherein the virtual product display fixtureincludes virtual inventory items from multiple vendors and wherein thevirtual inventory items are rendered according to the virtual planogram;and cause a display device to present the virtual market place relativeto the physical store fixture based on the display device's position andorientation.
 23. The server of claim 22, further comprising a planogrammanager interface configurable to allow a planogram manager to managethe virtual planogram.
 24. The server of claim 23, wherein the planogrammanager interface is configurable to provide at least one of thefollowing planogram management functionalities: create the virtualplanogram, update the virtual planogram, create product placement rules,create event triggering rules, and instantiate the virtual planogram.25. The server of claim 22, further comprising a vendor interfaceconfigurable to allow at least one of the multiple vendors to managetheir virtual inventory items.
 26. The server of claim 25, wherein thevendor interface is configurable to provide at least one of thefollowing vendor management functionalities: pay a planogram fee,purchase a planogram slot, virtually rotate stock, filing a request,store vendor account information, store product information, and setvendor preferences
 27. The server of claim 26, further comprising atransaction engine configurable to initiate a transaction associatedwith at least one of the virtual inventory items based on input receivedfrom a consumer via the display device.
 28. The server of claim 22,further configurable to model physics of the virtual market place. 29.The server of claim 22, wherein the physics is modeled in real-time. 30.The server of claim 22, wherein the position comprises a geographiclocation.
 31. The server of claim 30, wherein the geographic locationcomprises a GPS location of the display device.
 32. The server of claim22, wherein virtual planogram server is a web server located remotely tothe display device over a network.
 33. The server of claim 22, wherein amobile device comprises the virtual planogram server.
 34. The server ofclaim 22, wherein the display device comprises the virtual planogramserver.
 35. The server of claim 22, wherein the display device comprisesat least one of the following: a mobile phone, a tablet computer, atelevision, and a kiosk.
 36. The server of claim 22, furtherconfigurable to modify the virtual planogram based on a triggeringcondition.
 37. The server of claim 36, wherein the triggering conditionis based on at least one of the following: a news event, a userpreference, a time, a location, a user identity, a logical operator,sensor data, a temperature, a fee, a trend, a weather condition, and aprior purchase.
 38. The server of claim 22, wherein the virtualplanogram comprises an overlapping virtual planogram that layers overother virtual planograms.
 39. The server of claim 22, wherein thedigital representation comprises digital image data of the physicalstore fixture having physically present products.
 40. The server ofclaim 39, wherein the digital features comprise digital image features.41. The server of claim 40, wherein the image features include at leastone of the following: SIFT features, BRISK features, and vSLAM features.42. The server of claim 22, further comprising the planogram database.43. The server of claim 22, wherein the physical store fixture comprisesa physical store shelf.
 44. The server of claim 43, wherein the virtualproduct display fixture comprises a virtual store shelf.
 45. The serverof claim 22, wherein the virtual market place comprises an augmentedreality view superimposed on a real-world image within the displaydevice.
 46. The server of claim 22, wherein the virtual inventory itemsare incorporated into the virtual planogram based on a fee.
 47. Theserver of claim 46, wherein the virtual planogram comprises a feeschedule that includes the fee.
 48. The server of claim 47, wherein theschedule depends on at least one of the following factors: a purchase ofslot, a desirability of a slot, a time, a time period, a popularity of aslot, a dimension of the slot, a size of a slot, an exclusion of others,and geographic protocol.